Large-scale social-media analytics on stratosphere

The importance of social-media platforms and online communities - in business as well as public context - is more and more acknowledged and appreciated by industry and researchers alike. Consequently, a wide range of analytics has been proposed to understand, steer, and exploit the mechanics and laws driving their functionality and creating the resulting benefits. However, analysts usually face significant problems in scaling existing and novel approaches to match the data volume and size of modern online communities. In this work, we propose and demonstrate the usage of the massively parallel data processing system Stratosphere, based on second order functions as an extended notion of the MapReduce paradigm, to provide a new level of scalability to such social-media analytics. Based on the popular example of role analysis, we present and illustrate how this massively parallel approach can be leveraged to scale out complex data-mining tasks, while providing a programming approach that eases the formulation of complete analytical workflows

Life-Cycles and Mutual E_ects of Scientific Communities

AbstractCross-community e_ects on the behaviour of individuals and communities themselves can be observed in a wide range of applications. While previous work has tried to explain and analyse such phenomena, there is still a great potential for increasing the quality and accuracy of this analysis. In this work, we propose a general framework consisting of several di_erent techniques to analyse and explain cross-community e_ects and the underlying dynamics. The proposed methodology works with arbitrary community algorithms, incorporates meta-data to improve the overall quality and expressiveness of the analysis and identifies particular phenomena in an automated manner. We illustrate the benefits and strengths of our approach by exposing in-depth details of cross-community e_ects between two closely related and well established areas of scientific research. This work focuses on techniques for understanding, defining and eventually predicting typical life-cycles and events in the context of cross-community dynamics

UniStore: Querying a DHT-based Universal Storage

In recent time, the idea of collecting and combining large public data sets and services became more and more popular. The special characteristics of such systems and the requirements of the participants demand for strictly decentralized solutions. However, this comes along with several ambitious challenges a corresponding system has to overcome. In this demonstration paper, we present a light-weight distributed universal storage capable of dealing with those challenges, and providing a powerful and flexible way of building Internet-scale public data management systems. We introduce our approach based on a triple storage on top of a DHT overlay system, based on the ideas of a universal relation model and RDF, outline solved challenges and open issues, and present usage as well as demonstration aspects of the platform

UniStore: Querying a DHT-based Universal Storage

In recent time, the idea of collecting and combining large public data sets and services became more and more popular. The special characteristics of such systems and the requirements of the participants demand for strictly decentralized solutions. However, this comes along with several ambitious challenges a corresponding system has to overcome. In this demonstration paper, we present a light-weight distributed universal storage capable of dealing with those challenges, and providing a powerful and flexible way of building Internet-scale public data management systems. We introduce our approach based on a triple storage on top of a DHT overlay system, based on the ideas of a universal relation model and RDF, outline solved challenges and open issues, and present usage as well as demonstration aspects of the platform.peer-reviewe

Cost-Aware Processing of Similarity Queries in Structured Overlays

Large-scale distributed data management with P2P systems requires the existence of similarity operators for queries as we cannot assume that all users will agree on exactly the same schema and value representations and data quality problems due to spelling errors and typos. In this paper, we present an approach for efficient processing of similarity selections and joins in a structured overlay. We show that there are several possible strategies exploiting DHT features to a different extent (i.e., key organization, routing, multicasting) and thus the choice of the best operator implementation in a given situation (selectivity, data distribution, load) should be based on cost information al- lowing the system to estimate the computation and communication costs of query execution plans. Hence, we present a cost model for similarity operations on structured data in a DHT and demonstrate the efficiency of our proposal by experimental results from a large-scale PlanetLab deployment.peer-reviewe

Quality-driven resource-adaptive data stream mining?

Data streams have become ubiquitous in recent years and are handled on a variety of platforms, ranging from dedicated high-end servers to battery-powered mobile sensors. Data stream processing is therefore required to work under virtually any dynamic resource constraints. Few approaches exist for stream mining algorithms that are capable to adapt to given constraints, and none of them reflects from the resource adaptation to the resulting output quality. In this paper, we propose a general model to achieve resource and quality awareness for stream mining algorithms in dynamic setups. The general applicability is granted by classifying influencing parameters and quality measures as components of a multiobjective optimization problem. By the use of CluStream as an example algorithm, we demonstrate the practicability of the proposed model